Image stabilization techniques have been used in a wide variety of different applications, including surveillance applications, vehicle-mounted image sensor applications, robotics applications, and consumer electronics applications. Image stabilization is the process of eliminating or at least reducing the effects of unwanted image sensor motion (e.g., unwanted image sensor vibration or irregular image sensor motion) on pictures or video sequences. Many image stabilization approaches attempt to eliminate or at least reduce the amount of unwanted image sensor motion relative to a scene while preserving any intentional image sensor motion. In this regard, these image stabilization techniques synthesize a new image or video sequence from the perspective of a stabilized image sensor trajectory.
Among the primary classes of image stabilization techniques are mechanical image stabilization methods, electromechanical image stabilization methods, optical image stabilization methods, and electronic image stabilization methods. Mechanical image stabilization systems attempt to dampen the motion of the image sensor (or just the lens/image sensor subsystem). Electromechanical image stabilization systems detect motion of the image sensor and alter the position or orientation of the image sensor to offset the detected image sensor motion. Optical image stabilization approaches stabilize the image of the sensor by displacing the image as it is formed by the lens system in a way that offsets image sensor motion. Electronic image stabilization techniques involve modifying the captured images in ways that makes the captured images appear to have been captured by a more stable image sensor.
In some common electronic image stabilization techniques, the motion of an image sensor is estimated and mapped to a model of unwanted image sensor motions (e.g., unwanted jittering, panning, zooming, and tilting of the image sensor) over the time images are captured. Each image frame is transformed based on the image sensor motion model to generate a synthesized video sequence that is stabilized relative to the image sensor motion model. In many of these image sequence stabilization approaches, the image sensor motion model describes the motion of the image sensor relative to a spatially dominant component of the scene that is assumed to correspond to a stationary component of the scene, such as the background. Oftentimes, however, the spatially dominant component of the scene corresponds to a moving object (e.g., a child running in front of the image sensor), in which case the motion model describes the motion of the image sensor relative to a moving object, leading to undesirable stabilization results.